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INTRODUCTION 


This report stimmarizes progress of a research program whose purpose 
is investigation of appropriate modes of interaction between a human pilot 
and automated on-board decision making systems. Most of our efforts thus 
far have been concentrated on determining how to allocate decision making 
responsibility between human and computer. However, we have also initiated 
research on the topic of pilot problem solving in automated and semi- 
automated flight management systems. 

In the next section of this report, Rex Walden discusses the 
experimental situation he has developed for study of allocation of 
responsibility between human and computer. He also discusses experiments 
in progress that are aimed at determining various pilot performance 
parameters with varying degrees of automation. 

In the section following Walden's, Yee-Yeen Chu considers optimal 
allocation of responsibility between human and computer. He discusses various 
approaches and presents some theoretical results found in the literature. 

These results offer valuable insights into the allocation of decision making 
responsibility in flight management. 

In the following section, Joel Greenstein discusses efforts aimed 
at developing a model of hvrnian decision making in a multi-task situation 
that, abstractly at least, is very similar to that being studied by Walden 
and Chu. The models developed with this work should be directly applicable 
to the design of computer aids for allocation of decision making 
responsibility between human and computer. 

In the final section of this report, the pilot as a problem solver 
is discussed. The design of displays, controls, procedures, and computer 
aids for problem solving tasks in automated and semi -automated systems is 


considered. 
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PILOT DECISION-MAKING IN A FLIGHT MANAGEMENT SITUATION WITH VARIOUS 
DEGREES OF AUTOMATION* 

I. Introduction 

An experimental situation for study of pilot interaction with 
an automated airborne decision-making system has been developed. A digital 
computer simulation of an aircraft uses a CRT graphics system to present a 
cockpit-like display to an experimental subject "pilot". (See Figure 1.) 

The display includes standard aircraft instruments: artificial horizon, 
altimeter, heading and airspeed indicators, and a clock. Also displayed 
is a map which indicates the course the airplane is to follow. An airplane- 
shaped symbol indicates the airplane's actual position. A small circle moves 
along the mapped course and indicates the position the aircraft should have 
for it to be on course and schedule. Near the lower edge of the display, 
several dials are shown, which represent indicators for such things as fuel, 
electrical, or hydraulic subsystem status. The dial pointers move with 
dynamics provided by passing a zero-mean, Gaussian white noise input through 
a second-order filter. 

II . The task 

The pilot controls the aircraft pitch and roll dynamics with a 
joystick. Another control stick controls the airspeed. The pilot's task 
is to fly the airplane along the mapped route, maintaining a fixed altitude 


* 


This section is based on the M.S. thesis proposal of R. S. Walden. 
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Figure 2: Display When Pilot Has Reacted to a Possible Malfunction 
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and stable pitch and roll attitudes. An autopilot feature is provided which 
causes the airplane position to coincide with the on-course indicator. 

While the autopilot is on, the aircraft is maintained in level flight, on 
course. Switching on the autopilot has the effect of resetting the airplane 
to the nominal, desired attitude and course, no matter what the current 
state of the airplane happens to be. 

In addition to flying the airplane, the pilot must monitor the 
subsystem indicators for possible subsystem malfunction (indicated when the 
needle points downward - see Figure 1). When the pilot thinks a subsystem 
has malfunctioned, he enters the corresponding number on a 4 x 3 numerical 
entry keyboard. The display shown in Figure 2 then appears. 

On this new display, the subsystem dials have been replaced with 
two rows of numbers, one labelled 'BRANCH', the other labelled 'STATE'. 

This corresponds to the first level of a checklist associated with the sub- 
system which was selected. The pilot looks for a branch with a state of 
'O' and enters the branch number on the keyboard. If the checklist for that 
branch continues, the next checklist level is displayed. When the end of 
the checklist is reached, the subsystem indicators reappear, with the 
malfunctioning system corrected. These actions simulate the checklist procedure 
a pilot might perform when correcting malfunctions in a real aircraft. 

III. Program description 

Simple aircraft dynamics are used. The pitch rate, roll rate, 
and airspeed, relative to aircraft-based coordinate system, vary linearly 
with control stick deflection. These linear and angular velocities are 
projected onto earth-based reference axes. The earth-reference position. 
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roll, pitch, and heading angles are updated 10 times per second from the 
corresponding derivatives, using a time interval equal to the update period 
(.1 second). 

Data samples are collected twice per second. Several things are 
recorded in each sample: values of instantaneous speed, heading, attitude, 
and position of the aircraft; the values of control stick inputs; status 
of the autopilot (on or off); elapsed time; and the status of each of the 
subsystem indicator dials. The pilot's keyboard responses to the subsystem 
indicator dials are also recorded. These data are stored on disk for later 
analysis . 

The simulation is written in BLISS-11 for a PDP-11/40 minicomputer. 
The 11/40 is interfaced to a CRT graphics display. The computer also has 
a disk unit, floating-point arithmetic hardware, and a multiplexed A/D 
converter which interfaces with the control sticks used by the pilot. The 
simulation program, along with associated programs such as graphics routines, 
requires 28K (16 bit) words of core for operation. Floating-point arithmetic 
is used in simulating the aircraft dynamics; integer arithmetic is used 
elsewhere for speed. 

IV. Experiments 

The first experiment will be run using the simulation control 
and monitoring tasks as described earlier. No computer aid will be 
present; the pilot will have full responsibility for flying the airplane 
and monitoring the subsystem indicators. Two variables will be manipulated 
in these experiments: the complexity of the control (flying) task, and the 
workload involved with the monitoring task. 
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Two levels of difficulty of the control task will be used. One 
level will be provided by normal en-route flight. A more difficult level 
will be produced by simulated terminal area "landing" maneuvers, where more 
attention must be given to steering the airplane over a more complex route. 

The workload involved with the monitoring task will be manipulated 
by changing the rate at which subsystem malfunctions occur. The subsystems 
malfunction in a random-appearing manner. (However, the malfunctions are 
actually prearranged.) By increasing the rate at which the malfunctions 
occur, the workload presented to the pilot is increased. Three levels of 
arrival rate of malfunctions will be used. 

From the data collected in these experiments, the response time 
to subsystem malfunction, the time to service the malfunction, and the control 
performance of the pilot can be obtained. The probability of a false alarm 
(pilot mistakes a functioning subsystem for a malfunctioning one), of a missed 
event (the pilot fails to detect a malfunction when it occurs), and of an 
incorrect action (the pilot makes a mistake in searching the checklist), 
can also be estimated. From the results of this experiment, the parameters 
of unaided pilot decision-making will be determined. 

A second experiment, designed on the basis of the experimental 
results anticipated above, will employ a simple, non-adaptive computer aid. 

The computer will use a static (i.e., situation independent) policy to 
determine when it is appropriate to aid the pilot. This experiment will 
be similar to the first experiment in design. The effectiveness of the 
non-adaptive computer aid can then be evaluated. Also, within this experiment. 
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we will consider the pilot's ability to detect that the computer aiding 
system has degraded in performance. In the next section, Chu will discuss 
approaches to deciding when the computer should aid the pilot. 
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APPROACHES TO ADAPTIVE COMPUTER AIDING 
I . The prob lem 

Considering the allocation of decision making responsibility in 
a pilot-computer decision making system, our previous simulation analyses 
(Rouse [75,76]) have shown that overall system performance can be enhanced 
if the computer's decision making strategy adapts to the current state 
of the aircraft, the pilot and the computer. In this section of this report, 
we will discuss results from the literature that are potentially of use in 
the design of an adaptive system. 

The overall pi lot -computer system can be characterized as 

follows ; 

1) The two decision makers (pilot and computer) are faced with a 
multi-task event detection and action implementation situation in which a 
set of monitoring and control tasks randomly demands attention. 

2) The decision making situation can be represented as a priority 
queueing system with two very different and imperfect servers and with 
imperfect communication between them. 

3) To enhance overall system performance, the allocation of decision 
making responsibility should be dynamic (i.e. situation dependent) while 
conflict between the two decision makers should be avoided. 

Given the decision making situation characterized in the above 
statements, several questions arise. These are concerned with defining and 
quantifying the state of the pilot-computer-aircraft system, and then, assuming 
the computer to be a copilot of sorts, determining how to schedule computer 
decision making (e.g., when to turn it on or off) as a function of system 


state. 
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As noted earlier, we have chosen to view the pilot-computer 
decision making situation as a queueing problem. Thus, the numerous results 
which we will review here come mainly from the queueing literature. However, 
before pursuing this review, we want to mention an alternative view of the 
decision making situation. Namely, the situation might be described as a 
stochastic control process with a controller structure adjusted dynamically 
for on-line estimation and control in an uncertain system environment. Such 
a representation of our system would require a knowledge of the underlying 
sequential decision models of each controller (i.e. each decision maker). 

In this control context, we might start with the control of linear systems 
subject to random disturbances as well as random system failures and the 
detection and correction of failures. Thus it is possible to model the pilot- 
computer decision making situation via the stochastic differential equation 
approach. However, to avoid going into the microscopic details of system 
state, we have chosen to view the problem as optimal control of a queueing 
discipline. The remainder of this section of this report reviews the litera- 
ture appropriate to this approach. 

II. Review of the literature 
II -A. Introduction 

The early works on the control of queues were essentially descriptive 
analyses of a set of plausible control policies from which "optimal policies" 
were selected by mathematical optimization techniques. More recently, 
researchers have begun to employ a Markovian decision model to solve queueing 
control problems. The state of the queueing system (e.g. the ntnnbers of 
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customers in the system, the status of service facilities) is observed at 
certain times (the decision epochs), a control action is taken and costs 
(e.g. entering prices, waiting costs, service costs, switching costs, servicing 
rewards) are incurred. The aim is to find a policy, a strategy for choosing 
successive actions, that will minimize the expected total system cost (average 
or discounted) over a specified time horizon (finite or infinite) . The typical 
solution approach is first to show the existence of an optimal stationary 
policy then to design an algorithm employing techniques such as dynamic 
programming, linear programming, policy improvement or value iteration for 
computing an optimal policy. While it would be interesting to pursue dis- 
cussion of these techniques with full analytical detail, instead we shall 
concentrate on summarizing the models that have been used and the results 
that have been obtained. 

It is usually convenient to categorize the system of interest by 
1) the system structure (e.g. M/m/s, M/G/1, GI/M/1), 2) the decision 
variables (e.g. service parameters (service rates, ntimber of servers, etc.), 
arrival parameters, queueing disciplines, operating time parameters). In 
the following section static models are discussed before dynamic ones since 
many dynamic control models cannot be solved separately from the static design 
models. Also, since quite a few of the server control models have served 
as bases for the development of corresponding priority control models, we 
will start section II -B with discussions of server control (mainly with 
server on-off) followed by discussion of service rate control. 

In section II -C we will discuss many interesting models for the 
control of arrivals. Section II-D considers various priority control schemes 
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which include a whole class of service control-based priority queueing systems 
and a couple of examples on dynamic priority assignment policies. Section II -E 
discusses interactions in decision and control environments and those in 
sequential decision and stochastic control contexts. 

II-B-1. Control of server 

Here the control action is to turn the server on or off at the 
service completions or at the customer arrivals when system is empty. Costs 
are charged for 1) switching (on-off), 2) servicing and 3) holding the 
customers . 

Blackburn [72] considers the problem of controlling a M/G/1 queue 
by turning the server on and off. He shows that the optimal stationary policy 
which maximizes expected discounted reward over an infinite horizon has a 
simple critical number characterization : (M,m) . This (M,m) policy is to 
provide no service if the system size is m or less, and to turn the server 
on when the size is greater than M. This result is quite similar to those 
obtained from inventory theory. Blackburn analyzes the problem as a Markov 
renewal decision process with constant switching costs and linear holding 
costs and shows that the m will be either 0 (that is to turn service off when 
the system is empty) or -1 (the server is never off) . His model allows for 
customer balking and reneging. 

For the M/G/1 queue with server on-off cost, server running cost 
and customer waiting cost, results based on the analysis of continuous time 
and discrete time by Heyman [68] and Magazine [71] also yield (M,m) as an 
optimal solution. Deb [76] extends the results to a situation of a batch 
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service of size up to Q, the control limits (M,m) are computed for M/M/1 
case with linear service time costs. 

Considering other than the Markovian decision point of view, 

Man [ 73 ] and Shaw and Hsuan [74] have separately considered the stochastic 
optimal control strategies for arrival rate regulated and service rate con- 
trolled systems with time varying input traffic demand. A set of continuous 
time state-space differential equations is derived, the maximal principle 
is applied and a two point boundary value problem is obtained. However the 
complexity in computation and implementation makes the approach formidable. 
Based on the arguments of the Bang-Bang control solution, the authors suggest 
that the "fixed threshold" policy would be an acceptable suboptimal solution. 

Shaw [72] in his work on optimal ramp control found an optimal 
on-off type two-level threshold controller with respect to minimizing the 
sum of queue length and customer rejection. In his later work [76], he 
further shows that the optimal customer rejection (diversion) to minimize 
total delay of the accepted customers and the rejected customers is of the 
control-limit fom for the M/G/1 case. An explicit expression for the optimal 
threshold for the M/M/1 case is given as either 0 or infinity depending on the 
traffic intensity and the relative importance of waiting loss and downtime 
loss in the maintenance-repairing context. 

Balachandran [73] has considered the same on-off policy with control 
measures determined by the unfinished work in the system (hence the approach 
requires that service time be known for customers in the system). This 
so-called "D-policy" is later proven by Balachandran [75] to be superior 
to the usual N-policies (which turns the sever on when the queue size reacher 
the value N) for exponential service distribution with decreasing failure 


rates . 
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II -B -2. Controls on service rates 

In this case a service rate can be chosen from a set of the allowable 
service rates at customer arrivals or at service completions. 

Crabill [72] has employed a continuous time Markovian decision 
model to investigate the queueing situation with a finite number of exponential 
service rates each with associated cost rate r^. An optimal long run 
average cost rate (including holding and service costs) stationary policy 
is such that for i > j, where is the optimal service rate when 

i customers are in the queue. The scheme is primarily a set of sufficient 
conditions for eliminating from consideration certain service rates and 
conditions for the existence of an optimal stationary policy. This agrees 
with the intuition that under certain cost conditions it is optimal to always 
use the fastest service rate. Crabill [74] extends the previous model to 
a maintenance system considering costs depending on service rates and costs 
due to lost production. 

Lippman [75] generalizes previous results by implementing a cost 
structure which consists of a customer holding cost L, a reward for service 
completion and a service cost rate which increases with the service rate 
employed. He shows that the optimal service rate (when the discount factor 
is Z, when there are I customers in the system and N transitions remaining) 
is an increasing function of I, N, and L/Z. To conclude this section, 

Crabill [72,74] shows that there exists a monotone optimal average cost 
policy and extends the result to include switching costs and production 
losses. Lippman [75] establishes the existence of monotone optimal dis- 
counted and average cost policies. 
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II-C. Control of arrivals 

There are many contexts, such as the admission scheduling and 
routing in a hospital, in which the service rate is not the principal opera- 
tional problem. Quite recently the optimal arrival control models have been 
developed in a rather ad-hoc way. In this section we will discuss four 
scattered categories on control of arrials with the first two categories 
closely related to the priority assignment models discussed in the next 
section. 

The first category is an extreme control, accepting or rejecting 
each customer. This amounts to changing the arrival rate from its normal 
level (accept) to level zero (reject) . The second category is based on 
varying the arrival rates in an intermediate manner. The third category is 
based on optimal queue formation. The forth category is that in which the 
customers, rather than the queue, make the decision of accepting or refusing 
entry to the queue. 

Not allowing waiting lines or queues (in the first category - the 

extreme control) , Keilson [70] has considered an M/G/1 queue with arrivals 

accepted or rejected. Each type j customer entering the system offers a 

reward R. for being served and a service time distribution with mean rate 
J 

p,.. Observing these along with the number N of customers in the system, the 
arrival can either be accepted or rejected. Cost are charged for holding 
the customers in the system. The optimal average (and discount) cost policy 
over infinite horizon is found to be: Accept a type j customer if and only 

if l^jR- ^ Q> where Q > 0 is easy to find. Various versions of this problem 
have been analyzed by others with the result that certain p.-R relations lead 
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to similar optimal monotone policies. In the next section, we will also 
discuss stable customer pa3mient policies for purchasing a priority assignment 
in a queueing system (Balachandran [72]). 

In the second category (the intermediate arrival rate control), 
for the problem of an M/m/s queue with variable arrival rates and finite 
waiting places. Low [74] shows the existence of an optimal monotone 
average cost policy, i.e. an optimal arrival rate is a nonincreasing function 
of the number in the system. He also gives an algorithm for computing this 
policy. Kakalaic>:[69] has . considered an M/G/1 queue of J streams of customers 
with arrival rate chosen from the interval [0,a]. For each class there is 
a rejection cost, an acceptance reward and a holding cost. The author finds 
that the optimal policy for optimal average cost is of the bang -bang type; 
that is to accept all customers of class i (rate a) or accept none (rate 0). 

This indicates that the optimal policies of the first category satisfy a 
more robust set of conditions than originally specified. 

In the third category (the optimal queue formation), Nazarov [76] 
considered the G/M/2 queue with limited waiting places. He proposed an 
optimal dynamic (state-dependent) strategy of assigning arrivals to a service 
channel for a given functional cost structure which includes cost of holding 
customers and cost for loss of customers. The decision is determined by the 
function d(i,j), the conditional probability of assigning the arrival to the 
first facility given that the system has i customers in channel 1 and j 
customers in channel 2. Modeled as a linear Markov process and approached 
by the dynamic programming and finite difference equation technique, the final 
results are obtained through a successive approximation to minimize the expected 
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cost function (in a limiting average form). The implementation on a digital 
computer shows that dynamic formation of the queue, as compared to the conven- 
tional static approaches, results in a cost saving of up to 30%. The author 
suggests the use of the dynamic strategy for loading factor (utilization) 
near unity. For a total loading factor considerably different from unity 
the gain is insignificant. 

Considering the question of customer control over entering a queue 
with holding cost c or refusing to enter the queue with lost-service cost r 
(in the fourth category - the arrival joining/rejection) , Noar [69] gives 
optimal joining rules for a M/M/1 queue: join the queue if and only if the 

queue size N is less than or equal to M for self optimization (customer 
himself) and m for social optimization (customer + system) where in general 
M > m > 0. Thus self optimization leads to a more congested system than that 
of social optimization, as we might have expected. 

II-D. Control of queue discipline - Priority models 

The separation of customers at a service facility into distinct 
classes, and then servicing the classes according to the measure of importance 
gives rise to what is known as a priority queue. Jaiswal [68] gives an 
excellent discussion of the basic structure of priority queues. As Bronshteyn 
and Rykov [65] have pointed out, systems with optimal series of priorities 
are better than those without priority assignments, although the latter is 
better than the system with priorities inappropriately established. To 
optimize a priority queue, we might want to determine the assignments of 
customers to priority classes, or given a natural set of classes to determine 
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the service discipline so as to optimize some measure of system effectiveness. 

Most of the priority control papers have evolved as an outgrowth 
of classical descriptive models which are static (exogenous or state-indepen- 
dent) and deterministic (in the decision criterion) . Several of the arrival 
control approaches to priority assignment given in the previous section fall 
into this category and they will be discussed in the last part of this section. 
First we will examine the classical results of Cox and Smith [61], developing 
the models in parallel with the generalizations on dynamic scheduling disci- 
plines. Secondly, we will discuss several dynamic priority models in the 
context of time -dependent or state-dependent categories, and thirdly, the 
control of server disciplines in priority queues. 

In an M/G/1 nonpreemptive priority queue of finite J flows of 
customers with waiting cost c^ and service rate , Cox and Smith [61] 
have obtained an optimal priority assignment policy (called the jj--c solu- 
tion) which states: of all the possible nonpreemptive work-conserving 

stationary queueing policies, the head-of-the-line discipline with the highest 
priority assigned to the class j customers with higher products is that 

which minimizes the average waiting cost. This result has been utilized 
in many simple priority models. Among other extensions of it is the 
Bronshteyn's [65] generalization of p-c solution to the preemptive discipline 
for an M/M/1 queue. Practically speaking the p,-c solution can be employed 
as an operating rule for priority assignment by implementing a partitioning 
scheme which includes a set of J+1 critical values. Considering the same 
M/G/1 head-of-the-line discipline with a finite number of priority classes, 

Beja and Sid [75] present two properties of an optimal priority partition: 
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1) The |J,-c products are monotonic on a set of optimally partitioned 
intervals. 2) It is optimal to assign strictly as many different priorities 
as are allowed. The authors further discuss the situations for different 
information sets on p. and c. If p or c is unobservable then, instead of 
|j,-c products, the estimates c/E['t^ |c=c3 or |jl E[c|u=p] is used for priority 
assignment . 

A simimary of results for application of the extended p-c solution 
to dynamic scheduling of a two-class M/G/1 queue is given by Harrison [76]. 

He shows that there is a modified static policy that is optimal in maximizing 
the expected service reward r^ minus holding cost c^ incurred over an infinite 
time horizon. The modified optimal policies are: 1) If then, instead 

of p-c, we have p-r policy., 2) If c^^ ^0, C 2 ^ traffic 

intensity of flow-2) > 1 and P 2^2 ^ then serve class 2 customers or idle. 

3) If C 2 > c^=0 and C 2 ^ (rj^(l-X 2 /p 2 )) /(2E(t^ ^^ 2 ^ then serve class 2 customers 
or idle. 4) If *^1^1~^2^2 solution does not apply 

to these degenerate cases; criteria based on the higher moments of service 
time distributions are required. 

There are quite a few papers that include discussions of dynamic 
priorities. Jackson [60] is one of the first to introduce the concept of 
the time -dependent priorities. Assuming that each class i customer is assigned 
on arrival a random urgency number a^, Jackson's [65] dynamic priority for 
this customer is given by a random variable p^(t) ~ ^ » where b 

is a constant common to all customers (namely 1) and t^ is the time of 
arrival. So a newcomer takes precedence over one in the queue if and only 
if the difference between the former's urgency ntmber and that of the latter 
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is larger than the time the latter has spent waiting. Actually we have a 
fixed priority type of queue where a customer's priority does not change 
relative to the other customers in the queue when he arrives. Under certain 
hypotheses, the author gives bounds for expected waiting times. Kleinrock's 
[67] time -dependent priority model assumes a proportionality between priority 
and waiting time for each customer; p^(t) = b^(t-t^). Thus, Kleinrock's 
model allows changes in relative priority. 

Another category of dynamic priority concepts was introduced by 
Vekierov [67] with the implication of state-dependent priorities. Rykov and 
Lembert [67] show for M/G/1 queues that the optimal dynamic priorities are 
simply the ordinary or static priorities. In another words, the p,-c solution 
is optimal in an essentially broader class of dynamic service policies. 
However this nice property will not hold for a system with a finite queue 
length, as we will see next. 

Mova and Ponomarenko [74] considered an M/M/c queue with finite 

r^ waiting places. There are n different flows of customers with Poisson 

arrival rates and c servers with same service rate . Denoting the 

system state for which there are i^ class j customers waiting for service 

by (ii,i-,...,i ), the service policy d. (i^ ,i„, . . . ,i ) is to choose a class 
i/ n jiz n 

j customer with probability d^ (i^^ ,i2 > . . . , i^) . The authors set up the state 
probability equations (a Markov process, stationary regime), apply the linear 
programming and necessary condition for the principle of optimality to obtain 
an' optimal ordering for servicing. The optimal d for a cost structure that 
includes waiting and lost customer costs in the state penalty form are found 


to be either 0 or 1, or more precisely. 




0 j s 

1 j = s , 
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where s = s (i_ , i„ , . . . , i ) is the state index indicating the system's current 
configuration. Determination of the s's involves solving a set of quasilinear 
equations. An example for an M/M/1/2 case is given. Under the assumption 
that = r^ = 1; = 0.3j X .2 = 0.5, p<=l the minimum cost service discipline 

obtained is to serve class 1 flow when (ij^,i 2 ) = (1,0), (2,0), (2,1) and to 
serve class 2 flow when (ij^,i 2 ) = (0»1)> (Oj^), (1,1), (1,2), (2,2). The 
results yield a benefit of a 4.25% decrease in cost compared to fixed priority 
assignment to class 2 flow. Thus, in this case the classical rule of 
assigning relative priorities (such as the |i,-c solution) is not optimal. 

It seems natural to expect that the success realized for control 

of the server (section II-B-1) might be extended to a priority queueing model. 

For the 2 priority classes M/G/1 queue with server on-off at the arrival- 

departure epochs. Bell [71,73] has proven the existence of an optimal average 

cost policy of the (a,b,c) type. This optimal strategy is to turn the server 

off only when the system is empty and to turn the server on the first time that 

+ b-n^ ^ c where a,b,c are nonnegative constants with a+ b > 0 while 

n^^ and n 2 are the numbers of class 1 customers and class 2 customers in 

the system. For the general m priority classes, the optimal control actions 

are simply characterized by the m-dimensional hyperplane of the form: 

a, .n, + a„-n„ + ... + a *n = c. The results hold for a general cost structure 
112 2 mm 

which includes holding, running and switching costs. 

For the two priority classes situation mentioned above, Tijms [74] 
derives an expression for the average number of the class i customers and 
determines the best (l,l,c)-, (l,0,c)- and (0,l,c)- policies with respect 
to an average linear cost criteria. These efforts are mainly an attempt to 
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fit the control of server model into the priority class situations and the 
(a,b,c) policies can be viewed as the counterparts of the (M,m) policies of 
Heyman [68]. 

Not satisfied with the "static" design of a priority rule. Bell 
[ 73 ] presents the optimal priority policies that allow for changing priority 
rules as a function of the system's current state. He considers an M/G/1 
n=2 case with holding costs h^ > h^ and the service option of FCFS (first 
come first serve) or promotion of class 1 customers further back in the queue 
with penalty R. Bell shows the existence of an e -optimal minimum average 
cost stationary policy. The optimal policy is; Serve any class 1 customers 
at the head of the line or to promote the last class 1 customer and to serve 
him next if a class 2 customer is at the head of the line and there is a 
class 1 customer in the queue with at least K class 2 customers in front of 
him. The K is determined by the relation K ^ (Ep-) / (hj^-h^) . The results can 
be extended to the case of different service time distributions between the 
classes. However if the penalty R is state dependent, which unfortunately 
is usually the case, the optimal decision in any state will typically depend 
on the precise waiting list of class 1 and class 2 customers in the state 
description. 

Balachandran [72] and Kleinrock [67] have considered models in 
which a customer's priority is assigned according to the amount he is willing 
to pay, higher pa 3 mient receiving higher priority (Balachandran calls it 
"purchasing priority", Kleinrock [76] calls it "bribery priority"). 
Balachandran develops methods to determine the conditions for stable payment 
policies in situations where optimal policies are not stable. He defines 
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the stable policy as a policy under which no individual customer can improve 
his situation by deviating from the stable policy, given that all other 
customers use the stable policy. He illustrates that an optimal average cost 
policy need not be stable. For M/M/1 queues he exhibits two rules which are 
stable under certain conditions. He also considers the case for which waiting 
time costs increase with time. 

Nair and Neuts [71] adopt a probabilistic approach to compare 
waiting times under three priority rules: FCFS, SPT (shortest processing 

time first) and LPT (longest processing time first) . The analysis relies 
on the theory of semi -Markov and renewal processes. The comparison is achieved 
essentially by the derivation of the Laplace-Stieltjes transform of the 
trivariate distribution of transient and steady state waiting times under 
each of the three rules. This approach makes it possible to answer the ques- 
tions like; What is the probability that the waiting time (as a random variable 
here) at t of a customer under SPT rule whose service time is x is less than 
that of a customer under FCFS. However the complicated results shown in this 
paper could be aTmajor drawback as far as the implementation on an adaptive 
system is concerned. 

II-E. Miscellaneous Issues - On design in decision and control environments 

As has been shown in this survey of the optimal design of queueing 
systems, the queueing approach turns out to be a very handy and powerful tool 
for modeling a decision and control system. There are several important issues 
concerning the interaction and interfacing between the queueing model and 
design environment that have to be coordinated or resolved before we can talk 
about the application and implementation of this powerful methodology. 
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In the preemptive situation the SRPT (shortest remaining processing time 
first) disciplines are employed in substitution for the SPT for the system to 
be optimal. 

Thus far we have been mostly concerned with optimal queueing systems 
operating over an infinite horizon with a homogeneous evolution of time. 
Another class that arises in a non-stationary environment is concerned with 
how to best operate the system over time with a given cost structure. 

Prabhu [74] has considered an M/G/1/FCFS system, employed the optimal stopping 
time concept and derived the optimal stochastic control theory. The optimal 
control is achieved by choosing the stopping time T that minimizes the overall 
cost over a given class of stopping times. Hsuan and Shaw [76] discuss 
the optimal dynamic control and repairman assignment policies for a linear 
stochastic system with Markov jump parameters. They formulate the problem 
•as a Markov decision process and present an elimination of equivalent policy 
technique which, incorporated with the control limit techniques developed 
earlier, makes their approach a little more promising. Man [73] derives 
an optimal control strategy for a M/m/s/N system with arrival rates con- 
trolled. The control system is dynamic and is capable of handling the time- 
varying input traffic demand in a way to employ the on-line traffic load 
measurement for updating the new optimal control policy. The system is 
therefore claimed to be less vulnerable to the uncertainties present in the 


system parameters. 
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HUMAN DECISION MAKING IN MULTI-TASK SITUATIONS* 

I. The problem 

There are many situations in which the human decision maker is 
responsible for several tasks and must decide which of these tasks is to 
be performed. Industrial monitoring, air traffic control, aircraft piloting, 
and investment decision making are examples of such multi-task situations. 

As the complexity and performance demands of such situations increase, 
the human is faced with more tasks of greater variety. He may be unable 
to maintain acceptable performance with this increased workload. 

Attempts to reduce the human's workload and make more efficient 
use of his cognitive abilities are resulting in changes in the hioman's role 
in the operation of complex systems. He is no longer an integral part of 
many of the control loops within the system and instead devotes much of 
his capacity to monitoring machine controlled processes for events which 
require diversion of his attention to a particular process. In an air 
traffic control situation, for example, he may monitor the positions of 
several planes and divert his attention to a particular plane when it strays 
from its assigned path or begins its landing approach. 

A problem faced by the human in this situation is that the diversion 
of his attention to a particular task may cause him to miss the occurrence 
of events that relate to his other tasks. This problem might be alleviated 
by the addition of a computer aid with some event detection capability. 


This section is based on the Ph.D. thesis proposal of J. S. Greens tein 
entitled "Human Decision Making in Multi-Task Situations: Event Detection, 
Attention Allocation and Implications for Computer Aiding." This research 
is partially supported by the Joint Services Electronics Program. 
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If an interesting event were detected while the human's attention was 
diverted to another task, the aiding system could notify the human, or, 
if the aid had sufficient ability, it might make decisions without notifying 
the hxnnan. 

The use of a computer aid with responsibilities overlapping those 
of the himtian lessens the risk involved in the human's diversion of attention 
to one task. But such a responsibility overlap introduces the question 
of how decision making conflicts are to be avoided. A set of simulation 
studies by Rouse [75,76c] considered a situation in which human and computer 
monitored and acted with respect to a set of processes. It was found that 
the level of interaction in terms of what the human and computer knew about 
each other's actions had a significant effect on the niimber of conflicts 
and on overall performance. As feedback between the hximan and computer 
about each other's actions and planned actions was increased, conflict 
was reduced. An improvement in overall system performance was the result. 

A design question that arises, then, is that of devising some 
method of letting the human and computer know what each other is doing. 
Computer- to -human communication does not appear to be a serious problem. 

A simple status indicator could be used to inform the human of the tasks 
to which the computer is attending. 

Human- to -computer communication presents greater difficulties. 

If the human must consciously keep the computer apprised of his present 
and planned actions he may well find himself spending more time communicating 
his state than carrying out his tasks. The computer aid would not be doing 
much to reduce the human's workload in such a case. Instead, the computer 
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must have some way of inferring the human's state. That is, the computer 
needs some model of how the human detects events and allocates his attention 
among tasks. With this model the computer can determine its actions so 
as to minimize conflicts with the human. 

There is a need, then, for models of the human as an event detector 
and attention allocator in multi-task situations. These models would be 
of use both in the design of computer aiding systems and in their imple- 
mentation. The insights of the human decision making process the models 
provide would also be of use in the analysis and design of other systems 
in which the human has a decision making role. Some preliminary approaches 
to modeling the human in event detection and attention allocation tasks 
have been proposed by Rouse and Greens tein [76b] and will be discussed later 
in this paper. This research proposes to extend and validate these basic 
models. Specifically, it proposes to conduct an experimental investigation 
of human performance in event detection and attention allocation in a multi- 
process monitoring situation. The results of these experiments will then 
be used to validate and extend the preliminary models of the hxaman in such 
tasks. Finally, the implications of these models to the design and 
implementation of a computer aiding system will be considered. 

II. Related research 

Miller and Elkind [67] and Gai and Curry [75,76] have modeled 
the human in event detection tasks. Miller and Elkind consider a situation 
in which the human is a part of the control loop in a single task situation. 
They employ the idea that the human has an internal model of the controlled 
process which he uses to predict the effect of his control movements. 
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The detection of changes in process dynamics is based on the difference 
between the actual effect of his control movements and the effect his 
internal model predicts. Gai and Curry consider a situation in which the 
htanan is outside the control loop and monitors a process for failures. 

Their model of the hinnan employs estimation theory and decision theory and 
assumes that the monitored process can be represented as a Gaussian process. 
Failures are represented by a change in the mean of the monitored process . 

Several researchers have modeled the manner in which the human 
allocates his attention in multi-process monitoring tasks. Senders [64] 
employs an information theory approach while Carbonell [Carbonell, 66; 
Carbonell, Ward, and Senders, 68] and Senders and Posner [76] suggest 
queueing theojry approaches. Decision theory [Kvalseth, 75] and optimal 
control theory [Curry, Kleinman, and Hoffman, 75; Kleinman and Curry, 

76] have also been used to model the human in monitoring tasks. Smallwood 
[67] uses the internal model concept and proposes that the human relies 
upon internal models of the processes he is monitoring in making his atten- 
tion allocation decisions. 

All of these attention allocation models emphasize task monitoring. 
Their primary intent is to provide predictions of performance and workload 
for specific display designs . The attention allocation problem presented 
in this paper is different in that it is less concerned with the allocation 
of attention among monitoring tasks than with the allocation of attention 
between task monitoring and action with respect to events observed during 
task monitoring. Action with respect to an event is assumed to require 
the human to divert his attention from task monitoring for some period of 
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time. It is also assumed that all of the tasks can be monitored simulta- 
neously in the sense that the human updates his knowledge of all tasks 
whenever he chooses to monitor. 

This proposal suggests that models of hximan decision making might 
be used in computer aiding systems to reduce the human's workload in multi- 
task situations. Freedy and his colleagues [Preedy, et. al., 71; Freedy, 
Weltman, and Lyman, 72; Freedy, Weisbrod, and Weltman, 73; Weisbrod, Davis, 
and Freedy, 75; Steeb and Freedy, 76] have developed adaptive computer 
aiding systems for single task situations in which the computer aid learns 
to perform a task by observing the human's performance in the task. When 
sufficiently confident of its ability, the computer assumes control of 
the task from the human. Their research is directed toward developing 
human factors criteria for such shared decision making. Also concerned 
with the development of human factors criteria for the design of computer 
aiding systems are the experimental investigations of Foul ton and his 
colleagues [74]. They study the effects of various methods of computer 
assistance on human performance in sonar signal detection and classification 
tasks . 

Rouse [75, 76b, 76c] has investigated the issues arising when human 
and computer interact in multi-task situations. He considers situations 
in which the human and computer have overlapping abilities and responsi- 
bilities and shows, among other things, that the amount that the human 
and computer know about each other's actions has a significant effect on 
overall system performance. It is the intent of the research proposed 
in this paper to facilitate human- to-computer communication in such multi- 
task situations through the development of models of the htnnan's performance 


in such tasks . 
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III . Proposed approach 

There are several possible approaches to the modeling of the 
human in event detection tasks. Signal detection theory might be employed 
[sheridan and Ferrell, 74] but by itself it is of little use to a computer 
aid in explaining how the human actually detects events. If the task state 
can be modeled as a stochastic dynamic system an estimation theory approach 
[Cai and Curry, 76; Rouse, 76a; Willsky, 75] would be appropriate, but many 
realistic multi-task situations would not yield easily to such analysis. 
Discriminant analysis Lxatsuoka, 71; Afifi and Azen, 72] appears to offer 
advantages as an approach to modeling event detection. It offers the 
possibility of allowing the computer aid to adaptively define what the 
human decision maker considers to be an event. It might also allow the 
costs of errors to be inferred from the htjman's actions. Thus, a computer 
aid using a discriminant analysis technique mi^t analyze the human's 
decisions over a period of time and assist with the event detection tasks 
when sufficiently confident of its own ability. 

The problem of attention allocation in multi-task situations 
might be modeled as a system in which various tasks randomly demand the 
attention of the decision maker with various frequencies and for various 
durations. Additionally, some tasks would be expected to be of greater 
importance than others. If minimization of the decision maker's delay in 
acting upon events, particularly those relevant to important tasks, is a 
reasonable measure of performance, then a queueing theory approach may be 
useful in modeling the human as an attention allocator. 



-35- 


Some preliminary models of human performance in event detection 
and attention allocation tasks have been developed using the approaches 
mentioned above. They are discussed in some detail in an earlier paper 
[Rouse and Greenstein, 76b] . These models indicate that in event detection 
tasks the a priori probabilities of event occurrences, the values of correctly 
responding to events and not responding to non-events, and the costs of 
making false alarms and missing events are important parameters. In multi- 
task situations in which the decision to react to an event requires the 
human to divert his attention to a specific task for some period of time, 
the decision making process is considerably more complicated. Important 
parameters in this situation include the a priori probabilities of event 
occurrences in the different tasks, the amount of time attention must be 
diverted when responding to events in the different tasks, and the costs 
of delays in responding to events in the different tasks. Another important 
parameter in this situation appears to be the human's planning horizon, 
the number of actions or units of time that the human plans ahead. 

A set of two experiments has been planned to validate the 
proposed models of event detection and attention allocation in a specific 
situation. The first experiment, recently completed, investigates the 
human's event detection behavior, while the second experiment focuses 
on his attention allocation behavior. The situation employed involves 
the simultaneous monitoring of several dynamic processes for the 
occurrence of abnormal events, a situation representative, for example, 
of monitoring tasks in complex industrial plants, advanced aircraft, or 
air traffic control. The design of the experimental situation is presented 
in an earlier paper [Rouse and Greenstein, 76a] . 
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Figure 1 illustrates the display observed by subjects in the 
first experiment. The display depicts the measured values of the outputs 
of nine processes over 100 sampling intervals. The processes had identical 
second order system characteristics with zero-mean Gaussian white noise 
inputs of identical variance. The displayed measurements of the process 
outputs were corrupted by additive zero-mean Gaussian white' noise sequences 
which normally had identical variance. An abnormal event in a process 
was defined by an increase in the measurement noise variance such that 
the signal-to-noise ratio for each measurement following an event occurrence 
was decreased to 957o of the signal-to-noise ratio of the previous measure- 
ment. Thus abnormal events became more and more pronounced with each 
successive measurement. 

After scanning the nine process histories, subjects were given 
an opportunity to key in the numbers of processes in which they had 
decided an abnormal event had occurred. They also positioned a cursor 
to enter their estimates of the times at which the events occurred. 

Upon completing their responses they were given feedback regarding the actual 
states of the processes they had keyed in. The display was then erased, 
current scores were given, any abnormal processes detected were returned 
to the normal state, and a new display depicting the process histories 
advanced 10 units in time was generated as illustrated in Figure 2. 

Dashed vertical lines were used to indicate to subjects when they last 
responded to each process. 

Subjects were allowed to respond to as many events as they 


thought had occurred. They were awarded points for their hits, receiving 
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Figure 2: An Updated Display 
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high scores for responding to events soon after their occurrence and 
lower scores for tardier responses. A fixed number of points was deducted 
for each false alarm. Subjects were allowed to study the displays as 
long as they wished, but any time taken beyond the first minute on each 
iteration reduced the number of points awarded for hits made on that 
iteration. 

Eight subjects were each given three trials spaced over several 
days. Each trial was 20 iterations long. The first and third trials 
given half the subjects were identical and had one event scheduled per 
iteration. The second trial scheduled the same events as the first and 
third trials, but also scheduled an additional event each iteration to 
permit study of the effect of a priori event probabilities on performance. 
The rest of the subjects were given these runs in different order so that 
the first and third runs had two events scheduled per iteration while 
the second run had one per iteration. 

The results of this experiment will be used to test the adequacy 
of a discriminant analysis approach to modeling event detection behavior. 
The time histories of the subjects' decisions will be analyzed using 
discriminant analysis techniques to determine model parameters and the 
resulting model predictions will be compared with the subjects' actual 
performance. The ability of the model to adapt to individual differences 
and time varying behavior will also be considered. 

A second experiment is planned to investigate the human's 
attention allocation behavior in a situation similar to the one employed 
in the first experiment. But whereas in the first experiment subjects 
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were allowed to respond to as many events each iteration as they thought 
had occurred, in this experiment they can choose one process in which 
they believe an event has occurred or they can choose none. The amount 
of time the process histories are advanced on the succeeding iteration 
is dependent upon the decision they make. The decision not to enter 
a process number represents a decision to continue monitoring all processes 
for events. The process histories will then be advanced only 10 units 
in time on the next iteration. The decision to respond to an event they 
believe has occurred in one of the processes represents a diversion of 
attention to that process. The process histories will then be advanced 
more than 10 units in time on the next iteration (the exact number 
depending upon the process chosen) thus delaying their opportunity to 
respond to other events . 

Although subjects may take action with respect to no more than 
one process each iteration, they will be asked to enter a string of 
proposed actions. The first action in the string of responses represents 
the actual decision (to continue monitoring or to check one of the nine 
processes for the occurrence of an abnormal event). The remainder of 
the string lists the actions subjects would make given that each of the 
previous actions in the string could not be taken at that time. This 
sequence will provide information on the extent of the subjects' planned 
actions . 

In this experiment various event occurrence probabilities, 
advances of process histories on next iteration, and costs for delays 
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in responding to events will be associated with each of the processes. 

The values of these parameters will be displayed with the process histories 
to allow subjects to utilize them in making their attention allocation 
decisions . 

The results of this experiment along with those of the first 
experiment will be used to validate and extend the proposed models of 
human performance in event detection and attention allocation in a 
multi-process monitoring situation. The implications of these models 
to the design and implementation of a computer aiding system will then 
be considered, perhaps through the use of simulation experiments in which 
the models are used with a rudimentary computer aid to improve its event 
detection performance and reduce its tendency to conflict with the human. 
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THE PILOT AS A PROBLEM SOLVER 

The discussion in the previous three sections of this report have 
focused on the pilot and computer cooperating in decision making. In this 
section, we want to discuss the pilot's role as a problem solver in 
automated and semi-automated systems. First, we will consider the pilot's 
problem solving tasks in current flight management situations and then, we 
will discuss how automation may result in the pilot having new and different 
problem solving tasks. 

If one reads the "emergency procedures" sections of aircraft 

•3U 

manuals," one finds that most of the procedures can be generalized as 
searches of tree-like checklists. Because emergencies allow little time 
for action, the pilot must memorize the various procedures and then perform 
them by rote. While he may have some written material available for 
reference, he can not afford investing too much time in reading. Thus, the 
pilot must be well-trained in the execution of emergency procedures. 

Other problem solving situations for the pilot include course 
charges due to degradation of aircraft readiness (e.g., a lost engine) or 
unexpected turbulence. Another example is when conditions change from IFR 
to VFR (e.g., on approach) and the pilot finds out that the aircraft is not 
where he thought it was. A further example might be when the pilot 
unexpectedly finds another aircraft in his vicinity. 

In all of the above examples, the pilot is highly trained to 
respond appropriately. Computer aiding might be of use to the pilot in these 


The manuals read were those for the F-4, T-37, and T-38. 
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situations if it could rapidly supply information necessary to successful 
solution of the problem. Also, the computer might perform many of the checklist 
procedures. Further, it could perhaps sort out the numerous warning signals 
that might be precipitated by the problem solving situation. 

It is quite possible that automation will eventually perform the 
tasks noted in the previous paragraph and it is interesting to consider the 
implications of such automation. To clarify the issues, let us discuss the 
pilot's problem solving role in an aircraft where all problem solving is 
automated. First of all, why would there be a pilot in an automated aircraft? 
The reason is quite simple. The computer may encounter a task (perhaps, for 
example, with odds of 1 in 10^) that it is ill-equipped to deal with 
(e.g., a hijacking) or the computer may simply fail. Then, the pilot may 
have to take over manual control or at least solve the particular problem 
that has stymied the computer. 

When the situation requires the pilot to be the problem solver, 
he is faced with the difficulty of not only dealing with the complexity that 
existed before automation but also the complexity of the automation itself. 
Further, if the complexity of automation increases as it becomes more versatile 
and reliable, then the pilot will increasingly be faced with the following 
dilemma. As the versatility and reliability of automation increases, the 
frequency of the pilot having to act as a problem solver decreases while the 
complexity of the task he must perform increases. 

To cope with this dilemma in the design of flight management systems, 
one will have to accept the fact that the pilot will no longer be experienced in 
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problem solving in the sense that he will not have had numerous replications 
of practice in solving each possible problem. This conclusion leads to the 
following question: Should displays, controls, procedures, and computer 

aiding for the inexperienced* problem solver be different than those for the 
experienced problem solver? 

The first step in our approach to answering this question has been 
to review the literature. A computer search of Psychological Abstracts yielded 
citations for 2000-3000 papers on human problem solving published over the 
past ten years. We chose to review somewhat over 300 recent publications. 

We have not yet organized this material into reportable form. At this 
point, we will limit our comments on this literature to one general statement. 
It appears that displays, etc. for inexperienced problem solvers should be 
different than those for experienced problem solvers. While the literature 
reviewed certainly does not provide any design guidelines, it does point out 
the most important hypotheses to be investigated in an experimental study of 
pilot problem solving in automated flight management situations. 

Our current efforts in this area are aimed at developing an experi- 
mental scenario that captures the essence of the problem solving tasks we have 
discussed above. Our goal is to provide empirical results upon which the 
design of problem solving displays, controls, procedures, and computer aids 
can be based. To conclude, in this section we have pointed out what we feel 
will eventually be a very important problem in flight management. In our 
next report, we hope to provide more information on the relevant literature as 
well as a discussion of completed and planned research efforts in this area. 

* 

We want to emphasize that the word "inexperienced" is being used to mean 
someone who may perhaps be well-trained in problem solving procedures but 
nevertheless has not encountered many of the specific problems that he may 
be asked to solve. 
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